Throttle functionality of haptic controller

ABSTRACT

Example implementations may relate to a haptic hand-holdable controller configured with throttle functionality. An example device may take the form of a haptic controller, which senses tactile information and provides force feedback. The haptic hand-holdable controller may implement a throttle where a motor varies feedback to the hand-holdable controller to simulate a throttle.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.14/968,535, filed on Dec. 14, 2015, which is incorporated herein byreference in its entirety and for all purposes.

BACKGROUND

Robotic systems may be used for applications involving materialhandling, welding, assembly, and dispensing, among others. Over time,the manner in which these robotic systems operate is becoming moreintelligent, more efficient, and more intuitive. As robotic systemsbecome increasingly prevalent in numerous aspects of modern life, theneed for robotic systems capable of working alongside humans becomesapparent. Therefore, a demand for such robotic systems has helped openup a field of innovation in controllers, sensing techniques, as well ascomponent design and assembly.

SUMMARY

Example implementations may relate to a controller system that includesa rotatable knob having one or more touch sensors and an inertialmeasurement unit. With this arrangement, the controller system may beconfigured such that a throttle grip on the touch sensors, incombination with a horizontal orientation of the controller, loads athrottle operational mode onto the controller. The throttle operationalmode may include both controller functionality, such as areturn-to-center function, and configurable controller output (e.g.,rotation of the knob and touch data received from the touch sensors maygenerate input data that represents intended control actions provided bya user holding the controller.

In one aspect, a controller system is provided. The controller systemincludes a rotatable knob coupled to a base. The controller system alsoincludes at least one motor that is operable to apply atorque-generating force to the rotatable knob and one or more touchsensors arranged to sense touch input on a surface of the rotatableknob. The controller system also includes a control system configured todetect throttle mode input and operate the controller in a throttlemode, where the throttle mode includes operating the at least one motorto affect the rotation of the rotatable knob to simulate a throttle.

In another aspect, a method is provided. The method includes detecting athrottle mode input on a haptic controller, where the haptic controllerincludes a rotatable knob coupled to a base, at least one motor that isoperable to apply a torque-generating force to the rotatable knob, andone or more touch sensors arranged to sense touch input on a surface ofthe rotatable knob. The method also includes operating the at least onemotor to affect the rotation of the rotatable knob in a throttle mode tosimulate a throttle.

In another aspect, a method is provided. The method includes detecting athrottle mode input on a haptic controller, where the haptic controllerincludes a rotatable knob coupled to a base, at least one motor that isoperable to apply a torque-generating force to the rotatable knob, andone or more touch sensors arranged to sense touch input on a surface ofthe rotatable knob. The method includes, in response to detecting thethrottle mode input, setting an initial position of the knob in relationto the base. The method includes detecting a movement of the knob inrelation to the base and operating the at least one motor to return theknob to the initial position.

These as well as other aspects, advantages, and alternatives will becomeapparent to those of ordinary skill in the art by reading the followingdetailed description, with reference where appropriate to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B illustrate an example robotic system, according to anexample implementation.

FIGS. 2A and 2B illustrate an example computing device, according to anexample implementation.

FIGS. 3A, 3B, 3C, and 3D illustrate an example hand-holdable controller,according to an example implementation.

FIG. 4 illustrates example communication links, according to an exampleimplementation.

FIG. 5 illustrates an example hand-holdable controller, according toexample implementations.

FIG. 6 is an example flowchart for simulating throttle functionality forthe example hand-holdable controller, according to exampleimplementations.

DETAILED DESCRIPTION

Example methods and systems are described herein. It should beunderstood that the words “example,” “exemplary,” and “illustrative” areused herein to mean “serving as an example, instance, or illustration.”Any implementation or feature described herein as being an “example,”being “exemplary,” or being “illustrative” is not necessarily to beconstrued as preferred or advantageous over other implementations orfeatures. The example implementations described herein are not meant tobe limiting. It will be readily understood that the aspects of thepresent disclosure, as generally described herein, and illustrated inthe figures, can be arranged, substituted, combined, separated, anddesigned in a wide variety of different configurations, all of which areexplicitly contemplated herein.

I. OVERVIEW

According to various implementations, described herein is a controllersystem having a throttle mode. In particular, an example embodiment mayinvolve a hand-holdable controller that includes a knob as well as touchsensors (e.g., a curved touchpad) coupled to the rotatable knob.Rotation of the knob and touch data received from the touch sensors maycollectively generate input data that represents intended controlactions provided by a user holding the controller. Further, a motor maybe configured to apply torque-generating force to the knob, so as toprovide haptic feedback. The controller system may be configured todetect a throttle mode input (e.g., the controller detects itsorientation is horizontal and receives a touch input, such as a throttlegrip). In response to detecting a throttle mode input, the controllersystem may operate in a throttle mode where, for example, the motor isconfigured to only allow rotation of the knob through some predeterminedangle. In another aspect of throttle mode, the controller system mayreturn the knob to its original position when released (e.g., like athrottle on a motorcycle).

A computing device, such as a tablet, may receive input data from thecontroller and may interpret the input data to determine specificintended operations of a robotic system. Upon processing the input datato determine a particular interpretation, the computing device may sendcommands to a robotic system (or to another device) in order to causethe robotic system to carry out intended operations of variouscomponents such as actuators coupled to joints, end effectors,appendages, speakers, and/or light sources, among others.

In some implementations, the throttle mode may include a return tocenter function. For example, the control system may apply areturn-to-center function, e.g., with a constant torque-generatingforce, immediately upon detecting movement and in the opposite directionof movement (e.g., such that a user feels a constant back-pressureagainst rotation of the knob during rotation) until the knob returns tothe center (or initial) position. Similarly, the control system mayapply the return-to-center function only after the disengagement of thecurved touchpad (e.g., such that a user does not feel any back-pressurewhile touching the curved touchpad).

In some embodiments, the throttle mode may include a damping function.For example, to increase the intuitive feel of the controller, avelocity-dependent (or acceleration-dependent) damping function may beused such that a motor resistance applied to resist rotation of the knobincreases (or decreases) based on the velocity with which the knob isrotating. This increase may be a linear increase, an exponentialincrease, or any other function. Additionally or alternatively, aviscous damping factor (or gain) may be used to further increase (ordecrease) the damping function.

In some embodiments, the throttle mode may include a sonic function. Forexample, a sonic function (e.g., a sound wave) may be produced by themotor by oscillating torque. By altering the number of oscillations persecond, the frequency of the sonic function can be affected. Theintensity (i.e., the sound per power unit of area) of the sonic functioncan be affected by increasing or decreasing power to the motor. Thethrottle mode may be configured such that a sound only occurs when theknob is in a certain zone (e.g., a range of positions from an initialposition). Similarly, the sound may only occur for a certain duration oftime upon entry to the zone and can only occur again by leaving the zoneand re-entering the zone. Additionally or alternatively, a gain may beapplied to increase or decrease the sonic function.

In some embodiments, the throttle mode may include a force-dependentfunction (e.g., derivative of force applied). For example, thecontroller may apply a force to the knob (via the motor) that variesbased on a force being applied to the knob. In some embodiments, theforce-dependent function may vary with the derivative of the force beingapplied to the knob. Additionally or alternatively, a gain may beapplied to increase or decrease the force-dependent function.

The throttle mode may also include metadata for any of the functions.For example, the gains or damping factors may be stored as metadata inmemory storage. Another example of metadata may include an angle rangeto which the operational mode applies. For example, the function mayonly apply to 180 degrees of rotation on the knob or to any other range.

II. ILLUSTRATIVE SYSTEMS

Referring now to the figures, FIG. 1A shows an example configuration ofa robotic system 100. Robotic system 100 may be any device that has acomputing ability and interacts with its surroundings with an actuationcapability and/or with ability to emit/generate physical phenomena suchas light and/or sound, among others. For instance, the robotic system100 may be a robotic arm, a humanoid robot, or a quadruped robot, amongothers. In other examples, robotic system 100 may define a vehicle, awatch, a washing machine, actuated blinds, a conveyer belt, a speaker,or a light bulb, among many other examples. Additionally, the roboticsystem 100 may also be referred to as a robotic device, a roboticmanipulator, or a robot, among others.

The robotic system 100 is shown to include processor(s) 102, datastorage 104, program instructions 106, controller 108, sensor(s) 110,power source(s) 112, actuator(s) 114, and movable component(s) 116. Notethat the robotic system 100 is shown for illustration purposes only androbotic system 100 may include additional components and/or have one ormore components removed without departing from the scope of thedisclosure. Further, note that the various components of robotic system100 may be arranged and connected in any manner.

Processor(s) 102 may be a general-purpose processor or a special purposeprocessor (e.g., digital signal processors, application specificintegrated circuits, etc.). The processor(s) 102 can be configured toexecute computer-readable program instructions 106 that are stored inthe data storage 104 and are executable to provide the functionality ofthe robotic system 100 described herein. For instance, the programinstructions 106 may be executable to provide functionality ofcontroller 108, where the controller 108 may be configured to instructan actuator 114 to cause movement of one or more movable component(s)116.

The data storage 104 may include or take the form of one or morecomputer-readable storage media that can be read or accessed byprocessor(s) 102. The one or more computer-readable storage media caninclude volatile and/or non-volatile storage components, such asoptical, magnetic, organic or other memory or disc storage, which can beintegrated in whole or in part with processor(s) 102. In someimplementations, the data storage 104 can be implemented using a singlephysical device (e.g., one optical, magnetic, organic or other memory ordisc storage unit), while in other implementations, the data storage 104can be implemented using two or more physical devices. Further, inaddition to the computer-readable program instructions 106, the datastorage 104 may include additional data such as diagnostic data, amongother possibilities.

The robotic system 100 may include one or more sensor(s) 110 such asforce sensors, proximity sensors, load sensors, position sensors, touchsensors, depth sensors, ultrasonic range sensors, infrared sensors,Global Positioning System (GPS) receivers, sonar, optical sensors,biosensors, Radio Frequency identification (RFID) sensors, Near FieldCommunication (NFC) sensors, wireless sensors, compasses, smoke sensors,light sensors, radio sensors, microphones, speakers, radar, cameras(e.g., color cameras, grayscale cameras, and/or infrared cameras), depthsensors (e.g., Red Green Blue plus Depth (RGB-D), lasers,structured-light, and/or a time-of-flight camera), motion sensors (e.g.,gyroscope, accelerometer, inertial measurement unit (IMU), and/or footstep or wheel odometry), and/or range sensors (e.g., ultrasonic and/orinfrared), among others. The sensor(s) 110 may provide sensor data tothe processor(s) 102 to allow for appropriate interaction of the roboticsystem 100 with the environment. Additionally, the robotic system 100may also include one or more power source(s) 112 configured to supplypower to various components of the robotic system 100. Any type of powersource may be used such as, for example, a gasoline engine or a battery.

The robotic system 100 may also include one or more actuator(s) 114. Anactuator is a mechanism that may be used to introduce mechanical motion.In particular, an actuator may be configured to convert stored energyinto movement of one or more components. Various mechanisms may be usedto power an actuator. For instance, actuators may be powered bychemicals, compressed air, or electricity, among other possibilities. Insome cases, an actuator may be a rotary actuator that may be used insystems involving rotational forms of motion (e.g., a joint in roboticsystem 100). In other cases, an actuator may be a linear actuator thatmay be used in systems involving straight line motion. In either case,actuator(s) 114 may cause movement of various movable component(s) 116of the robotic system 100. The moveable component(s) 116 may includeappendages such as robotic arms, legs, and/or hands, among others. Themoveable component(s) 116 may also include a movable base, wheels,and/or end effectors, among others.

The above description of processor(s) 102, data storage 104, programinstructions 106, sensor(s) 110, and power source(s) 112 may apply toany discussion below relating to the respective component being used inanother system or arrangements. For instance, FIGS. 2A and 3A (amongother possible figures) illustrate processors, data storage, programinstructions, sensors, and/or power sources as being incorporated inother arrangement. These components at issue may thus take on the sameor similar characteristics (and/or form) as the respective componentsdiscussed above in association with FIG. 1A. However, the components atissue could also take on other characteristics (and/or form) withoutdeparting from the scope of the disclosure.

A robotic system 100 may take on various forms. To illustrate, refer toFIG. 1B showing an example robotic arm 118. As shown, the robotic arm118 includes movable component(s) 116 such as appendages 120A, 120B, and120C, among others. Additionally, the robotic arm 118 includes joints J1and J2, each coupled to one or more actuators (not shown) such asactuator(s) 114. The actuators in joints J1 and J2 may operate to causemovement of various movable component(s) 116 such as appendages 120A,120B, and 120C. For example, the actuator in joint J1 may cause movementof appendage 120B about axis 122 (e.g., resulting in rotation about anaxis of joint J1). Whereas, the actuator in joint J2 may cause movementof appendage 120C about axis 124 (e.g., resulting in rotation about anaxis of joint J2). Other examples may also be possible.

FIG. 2A is a block diagram showing components of an example computingdevice 200 that includes one or more processors 202, data storage 204,program instructions 206, power source(s) 208, sensors 210, display 212,and Input Method Editor (IME) 214. Note that the computing device 200 isshown for illustration purposes only and computing device 200 mayinclude additional components and/or have one or more components removedwithout departing from the scope of the disclosure. Further, note thatthe various components of computing device 200 may be arranged andconnected in any manner.

Display 212 may take on any form (e.g., LED, LCD, OLED, etc.). Further,display 212 may be a touchscreen display (e.g., a touchscreen display ona tablet). Display 212 may show a graphical user interface (GUI) thatmay provide an application through which the user may interact with thesystems disclosed herein.

Further, the computing device 200 may receive user input (e.g., from theuser of the computing device 200) via IME 214. In particular, the IME214 may allow for interaction with the GUI such as for scrolling,providing text, and/or selecting various features of the application,among other possible interactions. The IME 214 may take on variousforms. In one example, the IME 214 may be a pointing device such as acomputing mouse used for control of the GUI. However, if display 212 isa touch screen display, user touch input can be received (e.g., such asusing a finger or a stylus) that allows for control of the GUI. Inanother example, IME 214 may be a text IME such as a keyboard thatprovides for selection of numbers, characters and/or symbols to bedisplayed via the GUI. For instance, in the arrangement where display212 is a touch screen display, portions of the display 212 may show theIME 214. Thus, touch-input on the portion of the display 212 includingthe IME 214 may result in user-input such as selection of specificnumbers, characters, and/or symbols to be shown on the GUI via display212. In yet another example, the IME 214 may be a voice IME that may beused that receives audio input, such as from a user via a microphone ofthe computing device 200, that is then interpretable using one ofvarious speech recognition techniques into one or more characters thanmay be shown via display 212. Other examples may also be possible.

A computing device 200 may take on various forms. For instance, thecomputing device 200 may take the form of a desktop computer, a laptop,a tablet, a wearable computing device, and/or a mobile phone, amongother possibilities. To illustrate, refer to FIG. 2B showing an exampletablet 216. As shown, the tablet 216 includes touch-screen display 218that is configured to display a GUI and receive user-input such as byway of one or more touch gestures provided by a user of the tablet 216.Note that the tablet may also include other components not shown anddescribed herein.

FIG. 3A is a block diagram showing functional components of a haptichand-holdable controller 300, according to an example implementation.FIG. 3B is an illustration showing one possible implementation of ahand-holdable controller 300, which may include some or all of thecomponents shown in FIG. 3A. A haptic hand-holdable controller 300 mayalso be referred to herein as a hand-holdable controller, ahand-holdable-controller system, a controller system, a wirelesscontroller, or simply as a controller. In an example implementation, thecomponents shown in FIG. 3A may be part of a hand-holdable controllerwith a motorized knob, which can also receive input via a curvedtouchpad on its outer surface. Other implementations, which utilizeother components, are also possible.

In FIG. 3A, the hand-holdable controller 300 is shown to include one ormore processors 302, data storage 304, program instructions 306, powersource(s) 308, a base 310, a knob 312, sensors 314 such as touch sensors316, and a motor 318. Note that the hand-holdable controller 300 isshown for illustration purposes only and hand-holdable controller 300may include additional components and/or have one or more componentsremoved without departing from the scope of the disclosure. Further,note that the various components of hand-holdable controller 300 may bearranged and connected in any manner.

Base 310 may be arranged so as to allow a user to grasp onto (e.g.,hold) the hand-holdable controller 300 with one hand, while rotating theknob 312 with their other hand. Such a base 310 may be any shape, size,and/or form. Additionally or alternatively, the base 310 may be arrangedto be positioned on and/or coupled to a surface or a robot joint (oranother entity). With this arrangement, the user would not necessarilyhave to grasp onto the base 310 (e.g., so as to hold the controller 300)and could thus rotate the knob 312 with the controller 300 essentiallypositioned on and/or coupled to the entity. In a further aspect, thisbase 310 may be coupled to one or more other components of thehand-holdable controller 300, and/or may be integrated as part of acontroller housing (e.g., that extends into a center cavity in the knob312 such that the knob 312 can rotate about the portion of the housingthat extends from the base 310).

Rotatable knob 312 can take on various forms, such as the cylindricalform shown in FIG. 3B, or a conical form, among other possibilities.References herein to a “cylindrical” knob or other “cylindrical”components of the controller should be understood to encompasscylindrical, conical and other forms of the knob 312 and/or othercomponent. With such example arrangements, the controller 300 may bethus configured so that a user can provide input to the controller 300by way of rotating the knob 312 about (e.g., relative to) the base 310.For example, the degree and/or speed of rotation of the knob 312 mayprovide input for control of, e.g., a robotic device.

Further, the hand-holdable controller 300 may include one or moresensors 314 such as any of the example sensors discussed above in thecontext of the sensor(s) 110 of robotic system 100. For instance, thehand-holdable controller 300 may include touch sensors 316 such ascapacitive sensors, for example. The touch sensors 316 may be positionedand/or integrated within the knob 312 and/or within other components ofthe hand-holdable controller 300. For instance, the touch sensors 316may be arranged to detect touch on one or more surfaces of the knob 312.To do so, the touch sensors 316 could, for example, take the form of acurved touchpad arranged along at least a portion of the one or moresurfaces. With such example arrangements, touch data received via thesetouch sensors 316, such as during rotation of the knob 312, may be usedto control various aspects of the robotic system 100 (e.g., via thecomputing device 200) and/or various aspects of the computing device 200as further discussed below.

In an example implementation, such as that shown in FIG. 3B, thehand-holdable controller 300 may rotate about a central axis 326, andthe touch sensors may be arranged to provide a curved touchpad 320,which may also be referred to as a cylindrical touch surface. In FIG.3B, the cylindrical touch surface 320 is indicated by the crosshatchpattern on the surface of the knob 312. Further, in someimplementations, the cylindrical touch surface 320 can extend around theentire outer surface of the knob (or portions thereof), such that thetouch surface is a full cylinder (e.g., with no gaps in touch sensinganywhere in the circumference of the knob 312).

The hand-holdable controller 300 may additionally or alternativelyinclude other tactile sensors as well. For example, hand-holdablecontroller 300 may include any sensor that generates information arisingfrom physical interaction with the environment of the hand-holdablecontroller 300, such as capacitive sensors, positional feedback sensors,pressure sensors, proximity sensors, strain gauges, force sensors,temperature sensors, magnetic sensors, or others. For example, thehand-holdable controller 300 may include a proximity sensor (e.g., aHall-effect sensor or an infrared sensor) to detect the presence ofobjects near the hand-holdable controller 300 but that are not incontact with the hand-holdable controller 300.

In some implementations, the hand-holdable controller 300 may notinclude any mechanical or structural interface features (e.g.,mechanical buttons, switches, jacks, connectors, or controls), otherthan the knob 312. In such an implementation, the rotation of the knob312 and tactile or touch input may be the only forms of user input thatare possible via the controller 300. Alternatively, the hand-holdablecontroller 300 may include other interface features (not shown in theFigures) in addition to the knob 312. For example, the hand-holdablecontroller 300 may include a power switch or button, or other buttons,switches, jacks, connectors, or controls for providing input via thehand-holdable controller 300.

In an example implementation, the hand-holdable controller 300 mayinclude at least one motor 318 that is operable to applytorque-generating force to knob 312. The motor 318 may be a brushed DCmotor, a brushless DC motor, or an AC motor such as a synchronouselectric motor or an induction motor, among other possibilities.Additionally, the motor 318 may include a motor shaft, a stationarystator, and a rotor coupled to the motor shaft such that the motor shaftis configured to deliver mechanical power to, for instance, atransmission assembly, thereby causing a rotation of the transmissionassembly (which may be coupled to knob 312).

More specifically, the shaft of motor 318 may operably connected to theknob 312 and/or to a control component, such that the control componentcan receive an electrical input signal to control the rotation of theshaft (and thus the knob 312 as well). Alternatively, the knob 312 maybe connected directly to the control component (e.g., not by way of ashaft), among other possible arrangements. For example, a slip ring orrotary transformer may be used to couple electrical signals between twoparts that rotate in relation to each other, and thereby to power therotatable portion of the hand-holdable controller 300 (e.g., to rotatethe knob 312).

In a further aspect, the hand-holdable controller 300 may also include(i) potentiometers and/or variable capacitors that could be used forapplications such as determining a rotary position of the knob 312 asthe knob 312 rotates due to torque from the motor 318 and/or due to anexternal torque and/or (ii) a rotary switch that could be used to changeconfiguration (e.g., power on or off) of the controller 300 inaccordance with rotation of the knob 312 due to torque from the motor318 and/or due to an external torque, among other components.

With the above example arrangement, the at least one motor 318 iscontrollable in order to vary the amount, and possibly the direction, ofthe torque that is applied to the knob 312. In particular, motor 318 maybe operable to affect and/or resist rotation of the knob 312. Forinstance, the motor 318 may provide haptic feedback and/or change the“feel” of the knob 312 by applying torque to the knob in a clockwise orcounter-clockwise direction. By way of example, the motor may beoperable to, e.g., make rotation of the knob 312 by the user more orless difficult, to back drive a hand of a user holding the knob by wayof rotational feedback, to rotate the knob 312 without additional torquebeing applied by a user, to replicate the feel of detents or clicksduring the rotation of the knob, and/or to provide vibrational feedback,among other possibilities.

In a specific example, the controller 300 may control a joint of roboticsystem 100 (e.g., via computing device 200 as discussed below). In thisexample, the motor 318 could resist (or back drive) rotation of the knob312 in response to a determination (e.g., by the computing device 200)that a moveable component coupled to the joint is entering anon-permissible zone (e.g., unsafe zone), such as within a thresholddistance of a human for instance. Other examples are also possible.

As noted above, FIG. 3B shows an example implementation of ahand-holdable controller 300. As shown, the example hand-holdablecontroller 300 includes a base 310, a knob 312, and a motor (not shown)as well as any of the components discussed above in the context ofhand-holdable controller 300. The controller 300 may have a proximateend 325 a that is near the base 310 (illustrated in FIG. 3B near thebottom of the base 310) and a distal end 325 b (illustrated in FIG. 3Bnear the top of the knob 312). The knob 312 may rotate or be rotatedclockwise and/or counterclockwise about axis 326 in order to control arobotic system or a component thereof in various ways.

Further, touch data (or tactile data) may be received, during therotation of the knob 312 or while the knob 312 is stationary, from oneor more sensors (e.g., touch sensors 316 or tactile sensors) positionedon one or more surfaces of the knob 312. This touch data may affect themanner the robotic system 100 is being controlled. To illustrate, referto example FIGS. 3C-3D showing different hand positions on the examplehand-holdable controller 300.

FIG. 3C shows a hand 328A of a user grasping onto the base 310 of thehand-holdable controller 300 such as for the purpose of holding onto thehand-holdable controller 300. Whereas, the other hand 328B of the usergrasps onto the knob 312 such as for the purpose of providing user-inputby rotation and/or touch of the knob 312. As shown, the hand 328B graspsonto a relatively large surface area of the knob 312 such as by severalfingers as well as the palm on the surface area of the knob 312. Thetouch sensors may detect this particular touch gesture (e.g., this touchgesture may be referred to as a “full grip” or “full grasp”) by the userand may provide corresponding touch data representing this particulartouch gesture.

In contrast, FIG. 3D shows the hand 328A of the user grasping onto thebase 310 in the same manner as in FIG. 3C. However, in this case, theother hand 328B of the user grasps onto a relatively small surface areaof the knob 312 such as by placing only fingertips on the surface areaof the knob 312 close to the distal end 325 b. The touch sensors maydetect this different particular touch gesture (e.g., this touch gesturemay be referred to as a “fingertip grip” or “fingertip grasp”) by theuser and may provide different corresponding touch data representingthis different particular touch gesture. As such, the touch illustratedin FIG. 3D may result in different control functionality of the roboticsystem 100 (and/or the computing device 200) than the touch illustratedin FIG. 3C. Moreover, different touch gestures may result in differentcontrol functionality even if the characteristics of the rotation of theknob 312 (e.g., amount and/or speed of rotation) are the same acrossdifferent touch gestures and/or even if the component being controlledis the same across different touch gestures.

Many other example touch gestures (e.g., actions which may generatetouch data, such as gestures, grips, grasps, touches, and/or othertactile information) may also be possible without departing from thescope of the disclosure. For example, the hand 328A of the user maygrasp onto base 310 in the same manner as in FIGS. 3C and 3D. However,other touch gestures may include one or more of (i) a palming, (ii) apartial grip (with finger extension or retraction), (iii) a multi-fingersequence, (iv) a multi-touch, (v) a drag, (vi) a side surface hold,(vii) a side surface swipe, (viii) a fingertip only, (ix) a single tap(possibly at a certain location or within a certain area on the surfaceof the knob), (x) a double tap (possibly at a certain location or withina certain area on the surface of the knob), and/or (xi) a swipe or swipepattern (possibly at a certain location or within a certain area on thesurface of the knob), among other possibilities.

As one specific example, a palming grip may entail the palm of hand 328Bto be placed on the top of the knob 312 (e.g., at the top of theproximate end 325 a of hand-holdable controller 300). For example, anintuitive use of the palming grip may be as an indication of a stopcommand. Thus, the hand-holdable controller 300 may interpret touch dataindicative of a palming and issue a stop command to the computing device200 or robotic system 100 (or the hand-holdable controller 300 may sendthe palming touch data to the computing device 200, which in turn sendsa command to stop the robotic system 100 from performing an action or tostop an action that the robotic system 100 is currently performing).

In another example of touch input, a partial grip may be interpretedfrom touch data that indicates a touch gesture somewhere between thegrips illustrated in FIGS. 3C and 3D. For example, similar to the fullgrip show in FIG. 3C, all five fingers of hand 328B of the user may beused to grasp the hand-holdable controller 300 but, for the partialgrip, those fingers may be placed closer to the distal end 325 b (e.g.,above the dividing line 312A of the knob 312 illustrated in FIG. 3B). Inthe partial grip (although applicable to other grips as well), touchinput related to a finger retraction or finger extension may be used togenerate touch data. For example, sensors (such as touch sensors 316)may detect a finger retraction (e.g., one or more fingers of hand 328Bsliding or moving towards the distal end 325 b of hand-holdablecontroller 300) or a finger extension (e.g., one or more fingers of hand328B sliding or moving towards the proximate end 325 a of hand-holdablecontroller 300). This finger retraction or extension may vary thecommands sent to the robotic system 100. For example, a partial gripplus a finger extension may send control signals of increased magnitudeas the fingers extend further. Likewise, a partial grip plus a fingerretraction may send control signals of decreased magnitude as thefingers retract further. Other example touch gestures are possible andmay be programmable (e.g., via IME 214 or other hardware or software).

Alternatively, a partial grip may be defined in other ways. For example,a partial grip may be defined as a full grip minus one or more pieces oftouch input (e.g., touch input indicative of five (or less) fingers withno touch input indicative of a palm on top of knob 324).

In another example of touch input, a finger sequence may be used. Forexample, touch input indicative of the fingers of hand 328A being placedin a certain sequence may be used to generate touch data. For example,placing the five fingers down in a pattern may be identified and used.For example, a touch input indicative of the user touching the knob 324first with the thumb and then subsequently with each finger of hand 328a may be used to power the device on or off, or accomplish any otherfunctions. Likewise, any other finger sequence could be identified andused. For example, touch input indicative of a single finger tap (orthumb tap or palm tap) on any touch-sensitive surface could be used.Likewise, touch input related to a swipe could be used. For example, anindex finger of hand 328B may be placed on top of knob 324 and swiped ina pattern (e.g., a clockwise pattern) to generate touch data.

Touch gestures can be used in combination to vary the control signalssent to the robotic system 100. For example, a full grip being performedsimultaneously with a rotation of the knob 312 may actuate a joint at ahigh speed. By adding in a touch gesture (e.g., a fingertap) to the fullgrip and rotation, the control signal may be varied. For example, thespeed or magnitude of the control signal may be varied. Similarly, adifferent component may be controlled by the additional touch gesture(e.g., the fingertap may generate a control signal to close a gripper).

Other examples of touch input that may be used to generate touch datainclude, for example, a multi-touch (e.g., a combination of touches,such as a full grip followed by a palming, a drag (e.g., an identifiedgrip followed by a dragging motion), a side surface hold (e.g., twofingers of hand 328B placed and held alongside knob 312), and a sidesurface swipe (e.g., two fingers of hand 328B placed alongside knob 312and swiped in a clockwise manner). Of course, many other examples oftouch input are possible. Also, note that feedback (e.g., vibrationalfeedback, clicks, detents) could be provided by the controller 300 inresponse to transitions between such touch inputs.

Robotic system 100, computing device 200, and/or hand-holdablecontroller 300 may communicate with each other in various ways. Toillustrate, refer to FIG. 4 showing an example arrangement 400 includingcommunication links 402A, 402B, and 402C that provide for exchange ofinformation between the various systems. For instance, communicationlink 402A provides for communication between example hand-holdablecontroller 320 and tablet 216, communication link 402B provides forcommunication between tablet 216 and robotic arm 118, and communicationlink 402C provides for communication between robotic arm 118 and examplehand-holdable controller 320. Note that other arrangements may also bepossible as some communication links may be removed and othercommunication links may be added such as for communication with otherdevices not discussed herein.

Communication links 402A, 402B, and 402C may include wired links and/orwireless links (e.g., using various wireless transmitters andreceivers). A wired link may include, for example, a parallel bus or aserial bus such as a Universal Serial Bus (USB). A wireless link mayinclude, for example, Bluetooth, NFC, IEEE 802.11 (IEEE 802.11 may referto IEEE 802.11—2007, IEEE 802.11n—2009, or any other IEEE 802.11revision), Cellular (such as GSM, GPRS, CDMA, UMTS, EV-DO, WiMAX, HSPDA,or LTE), or Zigbee, among other possibilities. Furthermore, multiplewired and/or wireless protocols may be used, such as “3G” or “4G” dataconnectivity using a cellular communication protocol (e.g., CDMA, GSM,or WiMAX, as well as for “WiFi” connectivity using 802.11).

In other examples, the arrangement may include access points throughwhich the various systems may communicate with a cloud server. Accesspoints may take various forms such as the form of a wireless accesspoint (WAP) or wireless router. Further, if a connection is made using acellular air-interface protocol, such as a CDMA or GSM protocol, anaccess point may be a base station in a cellular network that providesInternet connectivity via the cellular network. Other examples are alsopossible.

In an example implementation, the hand-holdable controller 300 may beconfigured to receive instructions (e.g., from computing device 200)indicating an operational mode for the hand-holdable controller 300(e.g., for the rotatable knob 312), so as to essentially load theoperational mode onto the controller 300. In some embodiments, theoperational mode may be pre-programmed, may change or be set based ontouch input, or may change or be set based on other criteria (e.g.,based on sensor information such as inertial attitude data from aninertial measurement unit, gyroscope, and/or accelerometer). Such anoperational mode may define operational parameters of the motor (e.g.,motor 318) of the hand-holdable controller 300. As such, differentoperational modes may provide different “feels” to the knob by varyingthe haptic characteristics of the knob 312. In particular, different“feels” can be provided by varying the torque applied to the knob as itrotates and/or otherwise varying when and how torque is applied to theknob 312 and/or by varying the type (or type of control) of motor 318(e.g., by using a position rotation motor, a continuous rotation motor,a linear motor, etc.).

For example, a given operational mode may specify a specific amount ofturning resistance, or in other words, a specific amount of torque thatcounters rotation by the user (making it harder or easier for the userto turn the knob). In another example, an operational mode may specify arotationally-varying torque profile, which varies the amount ofresistance to turning as the knob rotates. In some embodiments, apositional rotation servomotor may be used where the torque rating ofthe servomotor at a particular position must be overcome to turn theknob. Other examples are also possible.

In another aspect, a given operational mode may specify a range ofrotation to which the knob 312 is restricted. To do so, an operationalmode may define the number of degrees of rotation from a baseorientation that are permissible in one or two directions. For example,an operational mode may limit rotation to within plus or minus 45degrees from a center point. Other examples are also possible.

In yet another aspect, a given operational mode may set limits on thespeed at which the knob can turn. For instance, a given operational modemay set a maximum or minimum number of degrees per second. Further, insome implementations, an operational mode may vary the maximum orminimum speed of rotation as a function of the number of degrees theknob has rotated from a base orientation.

In yet another aspect, a given operational mode may indicate whether ornot to apply a return-to-center function, which returns the knob to abase orientation when certain conditions are met. For example, areturn-to-center function may rotate the knob back to a base orientationwhenever input data from the touch sensors on the knob indicates thatthe user has released the knob. As another example, a return-to-centerfunction may only respond to release of the knob by rotating the knobback to the base orientation in certain orientations of the knob (e.g.,when the knob has been rotated by at least some threshold amount fromthe base orientation, or when the knob has reached a rotation limit).

In yet another aspect, a given operational mode may specify certainorientations or a certain range of rotation during which free spin ofthe knob should be allowed. In particular, when the knob is put in afree-spin mode, the motor may be disabled such that the knob is allowedto rotate freely about the stator of the motor. An operational mode mayalso specify certain trigger events that trigger the enabling ordisabling of free-spin mode. For example, an operational mode coulddefine a certain touch gesture or gestures that enable and/or disablethe free-spin mode. Other examples are also possible.

Other haptic parameters may also be adjusted or set by a givenoperational mode. For example, the hand-holdable controller may beconfigured to provide a variable resistance through customizable arcsizes of various sizes. As a specific example, a full (or partial)rotation of the knob could be divided into a variable number of arcs,and each arc could be of various sizes. Each of the variable number ofarcs could be defined to have a specific feel (e.g., one or moreoperational modes, such as resistance levels, speed, detents or nodetents, etc.). Other examples are also possible.

In a further aspect, an operational mode may also specify how touchinput that is received via a knob controller should be interpretedand/or translated into control signals for a robot system. For example,an operational mode may define one or more touch gestures that areavailable for use in the operational mode, and how these touch gesturesshould be interpreted. Various types of touch gestures may be definedand mapped to control functions, depending upon the particularimplementation.

In some cases, an operational mode may define one or more touch gesturesthat can be used to switch from the operational mode to one or moreother operational modes. Additionally or alternatively, touch gesturesthat place a knob controller into a given operational mode may bedefined globally, such that the controller can be placed into the givenoperational mode from any other operational mode. In either case, touchgestures may be used to vary the feel of the knob as it is rotated,and/or to vary manner in which rotation of the knob 312 is interpretedinto robot control signals. For instance, control signals sent viarotation of the knob may vary based on different manners in which a usergestures or grasps the knob 312 and/or may vary based on the location ofthe touch gesture along the one or more surfaces of the knob 312, amongother possibilities.

According to an example implementation, the hand-holdable controller 300may detect a rotation of the control knob (e.g., knob 312), and mayindicate that rotation of the knob to the computing device 200.Additionally or alternatively, the hand-holdable controller 300 mayprovide output data to a controller application running on computingdevice 200, which is indicative of detected touch data (e.g., duringrotation of the knob). As such, the computing device 200 may determinethe rotation of the knob 312 and/or touch gestures performed on the knob312, and may responsively generate corresponding control signals foranother device (e.g., robotic system 100) in accordance with therotation and/or detected touch.

To control a device, such as robotic system 100, the computing device200 may exchange messages with the robotic system 100 (e.g., viacommunication link 402B). The messages may include commands thatindicate the particular component to which the robotic system 100 shouldsend control signals. The messages may also include commands thatindicate the particular operations that should be carried out by theparticular component. As discussed, these particular operations arebased on an interpretation of the input data received by the computingdevice 200 from the hand-holdable controller 300.

Although reference is made throughout to a hand-holdable controller,this disclosure is not limited to hand-holdable controllers. Forexample, the controller 300 could attach to a vertical surface (e.g.,via a mechanical and/or magnetic attachment system). In another example,the controller 300 could attach directly to a device (or portion of adevice) it is controlling, such as a speaker, a robot arm, a motorcycle,a bicycle, a lighting system, or another controllable device. Thecontroller 300 could be configured to attach to an electric vehicle(e.g., an all-terrain vehicle, bicycle, motorcycle, etc.) as the solemeans of throttle control, thus providing anti-theft protection to theelectric vehicle (i.e., without a way to control throttle there is noway to control propulsion).

FIG. 5 shows a hand 328A of a user grasping onto the base 310 of thehand-holdable controller 300 such as for the purpose of holding onto thehand-holdable controller 300. Whereas, the other hand 328B of the usergrasps onto the knob 312 such as for the purpose of providing user-inputby rotation and/or touch of the knob 312. As shown, the hand 328B graspsonto a relatively large surface area of the knob 312 such as by one ormore fingers wrapping around part of the top surface area of the knob312, as well as the thumb wrapping around part of the bottom surfacearea of the knob 312. This grip may be generally referred to as athrottle grip. Other throttle grips may be possible. For example, thehand 328A could be reversed such that the one or more fingers wraparound part of the bottom surface area of the knob 312 and the thumbwraps around part of the top surface area of the knob 312. The curvedtouchpad (or touch sensors) may detect a throttle grip by the user andmay provide corresponding touch data representing this particular touchgesture. Additionally, the controller 300 (e.g., via the IME 214 orprogram instructions) may provide customizable or pre-programmedthrottle grips.

As described previously, the hand-holdable controller may includesensors 314, such as gyroscopes, accelerometers, magnetometers, andothers. In some embodiments, an specific operational mode, referred tothroughout as “throttle mode” may be applied for throttle functionality.In some embodiments, the throttle mode may be set, based at least inpart, on detection of a throttle grip. For example, the control systemmay detect a throttle grip and responsively load an operational mode tothe controller 300, such as a throttle mode, that simulates the feel ofa throttle and/or generates robot control signals corresponding to thefunction of the controller as a throttle.

Additionally or alternatively, the throttle mode may be set based onorientation of the controller 300. The orientation of the controller maybe determined from data received from sensor(s) 314, such as from aninertial measurement unit composed of one or more gyroscopes,accelerometers, and/or magnetometers. For example, these sensor(s) 314may detect the controller 300 being oriented horizontally with respectto the ground and, responsively, may load an operational mode to thecontroller 300, such as a throttle mode, that simulates the feel of athrottle and/or generates robot control signals corresponding to thefunction of the controller as a throttle.

The throttle mode may also be set based on a combination of the controlsystem (i) detecting a throttle grip on the controller 300 and (ii)detecting an orientation of the controller 300. For example, thethrottle mode may only be loaded to the controller 300 after both (i)detecting a throttle grip on the controller 300 and (ii) detecting thatthe controller 300 is oriented horizontally with respect to the ground.

The throttle mode may be configured to simulate the functionality of athrottle such as a motorcycle throttle (or to provide a more intuitivethrottle controller). For example, various vehicles (e.g., motorcycles,all-terrain vehicles, snowmobiles, jet skis, etc.) include inputcontrols such as throttles in the form of rotary twist-grip mechanisms.These rotary twist-grip mechanisms may be mechanically coupled tolinkages (or electronically coupled) that control a throttle plate orthrottle valve at an engine. The rotary twist-grip mechanisms may have amaximum travel in one or both directions of rotation via either aphysical stop mechanism or a spring mechanism. Rotating the twist-gripin one direction opens the throttle plate or throttle valve, increasingthe fuel provided to the engine. Upon release, the twist-grip returns toits initial position, e.g., via the spring bias.

In some embodiments, the hand-holdable controller 300 may include acontrol system configured to detect a throttle mode input. As usedthroughout this disclosure, a throttle mode input refers to what isdetected (e.g., touch data or sensor data) to put the controller in aspecific operational mode (i.e., throttle mode). An operational mode maydefine both (i) how the controller 300 behaves (e.g., whether it has asonic function, detents, return-to-center, and/or other functionality)and (ii) how user input (e.g., rotation of knob and touch gestures)should be interpreted and translated into control signals, e.g., robotcontrol signals.

After detecting a throttle mode input, the control system may load thespecific operational mode (e.g., the throttle mode functionality and howuser input should be interpreted) to the controller 300, the computingdevice 200, and/or other aspects of the robotic system 100. As describedpreviously, the functionality may be customizable and may include anyfunctions previously described, such as a return-to-center function, asonic function, a rotationally-varying function. The functionality mayapply to the entire 360 degrees or rotation of the knob 312 or may belimited to customizable arcs or ranges of rotation of the knob 312.

After the controller 300 detects a throttle mode input and the throttlemode is loaded to the controller, input data received via the controller(e.g., knob rotation or touch gestures) may then be interpreted togenerate system control signals in the manner specified by theoperational mode that is loaded. For example, input data received viathe controller 300 in throttle mode may be used to generate a controlsignal to open a throttle (or operate a component of robotic system 100)by a certain amount when the knob 312 is rotated in one direction, startor stop the cruise control function, or otherwise generate controlsignals for the robotic system 100.

In some implementations, the throttle mode input may include touch data(e.g., touch data corresponding to a throttle grip) received from thecurved touchpad 312 of the hand-holdable controller 300. In otherimplementations, the throttle mode input may include sensor data (e.g.,inertial data from an inertial measurement unit comprising one or moregyroscopes, accelerometers, and/or magnetometers). Alternatively, thethrottle mode input may be a combination of touch data and sensor data,such as a throttle grip and inertial data corresponding to thehand-holdable controller being oriented in a substantially horizontalposition.

Upon detecting the throttle mode input, the hand-holdable controller 300may be configured to operate in a specific operational mode, e.g., athrottle mode. The throttle mode, as well as functionality related tothe throttle mode, may be preconfigured (e.g., via computing device 200and IME 214), may be configurable (e.g., via touch input or programinstructions 306 at the hand-holdable controller or via computing device200), and/or may include various functionality as previously describedin reference to operational modes.

For example, the throttle mode may be an operational mode that specifiesone or more positions, such as an initial position, one or more stoppositions, and one or more interim positions. The initial position mayconsist of the position of the knob 312 of the hand-holdable controllerin relation to the base 310 of the hand-holdable controller 300 at thetime when the throttle mode input is detected. The positions may also bepreconfigured or configurable (e.g., the controller may detect a touchinput or touch gesture and change the position of the initial position,stop position, and/or interim positions). The stop positions may beprogrammed to be a certain distance (or number of degrees) in one ormore directions. For example, a stop position may be used to simulatethe maximum travel of a throttle (i.e., to simulate the end travel of aspring). Interim positions may be used to increase the realism of thethrottle mode, e.g., by simulating gear shifts or increasing thegranularity of damping, gains, or other functions.

In some embodiments, loading the throttle mode to the controller 300 maycause the controller to implement “return-to-center” functionality. Morespecifically, the throttle mode may cause the hand-holdable controller300 to rotate the knob 312 back to its initial position relative to thebase 310, when it is released by a user at position other than theinitial position. For example, the throttle mode may specify an initial(or base) position of the knob 312 relative to the base 310, or theinitial position may be set upon detecting the throttle mode input. Insome implementations, the initial position (and other positions) may bedetermined or set using position sensors, such as rotary encoders orHall effect sensors. The control system may be further configured todetect a movement of the knob 312 relative to the base 310 (e.g., via aposition sensor such as a rotary encoder or Hall effect sensor). Thecontrol system may be configured to apply torque-generating force to themotor 318 to return the knob 312 to the initial position.

The control system may apply torque-generating force in various manners.For example, the control system may apply a return-to-center function,e.g., with a constant torque-generating force, immediately upondetecting movement and in the opposite direction of movement (e.g., suchthat a user feels a constant back-pressure against rotation of the knob312 during rotation) until the knob 312 returns to the center (orinitial) position. Similarly, the control system may apply thereturn-to-center function only after the disengagement of the curvedtouchpad (e.g., such that a user does not feel any back-pressure whiletouching the curved touchpad).

In some implementations, the throttle mode may include areturn-to-center function. For example, the control system may apply areturn-to-center function for the knob 312, e.g., by applying a constanttorque-generating force via the motor 318. This force may be appliedimmediately upon detecting a movement of the knob 312 from the initialposition in a first direction, and in the opposite direction of movementof the first direction (e.g., such that a user feels a constantback-pressure against rotation of the knob during rotation) until theknob returns to the center (or initial) position. Alternatively, thecontrol system may apply the return-to-center function only after thedisengagement of the touch sensors (e.g., such that a user does not feelany back-pressure while touching the curved touchpad but the knob 312returns to center after release).

In some embodiments, the throttle mode may include a damping function.For example, to increase the intuitive feel of the controller 300, avelocity-dependent (or acceleration-dependent) damping function may beused such that the motor 318 applies resistance to rotation of the knob312. The motor resistance may increase (or decrease) based on thevelocity (or acceleration) with which the knob 312 is rotating. Thisdamping function may be linear, exponential, or any other function.Additionally or alternatively, a viscous damping factor (or gain) may beused to further increase (or decrease) the damping function.

In some embodiments, the throttle mode may include a sonic function. Forexample, a sonic function (e.g., a sound wave) may be produced by themotor 318 by oscillating torque. By altering the number of oscillationsper second, the frequency of the sonic function can be affected. Theintensity (i.e., the sound per power unit of area) of the sonic functioncan be affected by increasing or decreasing power to the motor 318. Thethrottle mode may be configured such that a sound only occurs when theknob 312 is in a certain zone or position (e.g., a range of positionsfrom an initial position). Similarly, the sound may only occur for aduration of time upon entry to the zone and can only occur again byleaving the zone and re-entering the zone. Additionally oralternatively, a gain may be applied to increase or decrease the sonicfunction.

In some embodiments, the throttle mode may include a force-dependentfunction (e.g., derivative of force applied). For example, thecontroller 300 may apply a force to the knob 312 (via the motor 318)that varies based on a force being applied to the knob 312. In someembodiments, the force-dependent function may vary with the derivativeof the force being applied to the knob 312. Additionally oralternatively, a gain may be applied to increase or decrease theforce-dependent function.

The throttle mode may also include metadata for any of the functions.For example, the gains or damping factors may be stored as metadata indata storage, such as data storage 304. Another example of metadata mayinclude an angle range to which the operational mode applies. Forexample, the function may only apply to 180 degrees of rotation on theknob 312 or to any other range of rotation.

In another aspect, the control system may oscillate torque through themotor at a specific frequency. This oscillation may produce a sonicoutput, or sound, such as a “tick” for one second at a certainfrequency. In some embodiments, this “tick” may give the effect ofbumping into metal. In some embodiments, the sonic output (and otherfunctions as well) may be configured such that it is used once for asmall duration of time when entering a window of positions (e.g., withina few degrees of the initial position or a stop position). In someembodiments, for the sonic output to be used again, the knob 312 mustleave the window of positions and re-enter the window of positions.

Gains, or damping factors, may also be used with one or more of thecontrol system functions. For example, the velocity-dependent torque maybe multiplied by a viscous damping factor to further increase therealism or intuitiveness of the throttle control (e.g., the viscousdamping factor may be used with the velocity-dependent torque to makethe controller feel like it is moving through mud or more difficult tomove). Other examples are also possible.

In some embodiments, further functionality may be used after loading inthrottle mode. For example, a touch input or touch gesture may be usedto provide various throttle functionality (or robot functionality). Asdescribed previously, the operational mode may define both (i) how thecontroller 300 behaves (e.g., whether it has a sonic function, detents,return-to-center, and/or other functionality) and (ii) how user input(e.g., rotation of knob and touch gestures) should be interpreted andtranslated into control signals, e.g., robot control signals.

In some embodiments, the throttle mode may further provide cruisecontrol functionality via the controller 300. For example, afterengaging in a throttle grip in a horizontal position, the knob 312 maybe rotated to simulate an open throttle and a thumb tap (or other touchinput) may indicate cruise control engagement. The control system mayset a cruise control knob position when the thumb tap occurs and beconfigured to operate in the throttle mode by keeping the knob 312 atsubstantially the cruise control knob position until the cruise controlis disengaged (e.g., by another touch input).

In another aspect, certain touch gestures may indicate otherfunctionality while in throttle mode. For example, the hand-holdablecontroller may implement a braking functionality while in throttle modeafter certain touch gestures (e.g., quickly tapping all four fingers onthe surface of the knob 312 while in a throttle grip or applying over athreshold amount of pressure on the surface of the knob 312). Becausethe operational modes are programmable, numerous other examples arepossible.

III. ILLUSTRATIVE METHODS

FIG. 6 is a flowchart illustrating a method 600, according to an exampleimplementation. In particular, method 600 may be implemented to simulatethrottle functionality on a hand-holdable controller.

Method 600 shown in FIG. 6 (and other processes and methods disclosedherein) presents a method that can be implemented within an arrangementinvolving, for example, the robotic system 100, the robotic arm 118, thecomputing device 200, tablet 216, hand-holdable controller 300, examplehand-holdable controller 320 and/or within the arrangement 400 shown inFIG. 4 (or more particularly by one or more components or subsystemsthereof, such as by a processor and a non-transitory computer-readablemedium having instructions that are executable to cause the device toperform functions described herein). Additionally or alternatively,method 600 may be implemented within any other arrangements and systems.

Method 600 and other processes and methods disclosed herein may includeone or more operations, functions, or actions as illustrated by one ormore of blocks 602-608. Although the blocks are illustrated insequential order, these blocks may also be performed in parallel, and/orin a different order than those described herein. Also, the variousblocks may be combined into fewer blocks, divided into additionalblocks, and/or removed based upon the desired implementation.

In addition, for the method 600 and other processes and methodsdisclosed herein, the flowchart shows functionality and operation of onepossible implementation of present implementations. In this regard, eachblock may represent a module, a segment, or a portion of program code,which includes one or more instructions executable by a processor forimplementing specific logical functions or steps in the process. Theprogram code may be stored on any type of computer readable medium, forexample, such as a storage device including a disk or hard drive. Thecomputer readable medium may include non-transitory computer readablemedium, for example, such as computer-readable media that stores datafor short periods of time like register memory, processor cache andRandom Access Memory (RAM). The computer readable medium may alsoinclude non-transitory media, such as secondary or persistent long termstorage, like read only memory (ROM), optical or magnetic disks,compact-disc read only memory (CD-ROM), for example. The computerreadable media may also be any other volatile or non-volatile storagesystems. The computer readable medium may be considered a computerreadable storage medium, for example, or a tangible storage device. Inaddition, for the method 600 and other processes and methods disclosedherein, each block in FIG. 6 may represent circuitry that is wired toperform the specific logical functions in the process.

At block 602, method 600 involves detecting throttle mode input on ahand-holdable controller, wherein the hand-holdable controller comprisesa rotatable knob coupled to a base, at least one motor that is operableto apply torque-generating force to the rotatable knob, and a curvedtouchpad arranged to sense touch input on a curved surface of therotatable knob.

In an example implementation, throttle mode input data may be receivedby computing device 200 (or robotic system 100) from hand-holdablecontroller 300 (e.g., via communication links 402A or 402C). Thethrottle mode input data may represent touch data received via touchsensors 316 of the hand-holdable controller 300 and/or sensor datareceived via sensors 314 of the hand-holdable controller 300. Moreover,this throttle mode input data may affect one or more aspects of thecomputing device 200 as previously disclosed.

Additionally or alternatively, the computing device 200 may receiveinput data representing a rotation of the knob 312 and process andinterpret this input data into one or more operations that should becarried out by one or more components of the robotic system 100. Thecomputing device 200 may then send commands to the robotic system 100(e.g., via communication link 402B) and the robotic system 100 may carryout these operations based on the received commands.

Various implementations may generally be discussed below in the contextof the hand-holdable controller 300 providing functionality of therobotic system 100 by way of the computing device 200 interpreting inputdata received from the hand-holdable controller 300. However, otherimplementations may also be possible. For instance, the hand-holdablecontroller 300 may control the robotic system 100 directly (e.g., viacommunication link 402C). As such, any functionality of computing device200 described herein may be incorporated within the hand-holdablecontroller 300. Other examples and implementations may also be possible.

In a further aspect, the received input data may be in the form ofcomputer-readable data packets, among other possible forms.Additionally, the input data may be received continuously (e.g., inreal-time) or may be received from time-to-time (e.g., periodically).Further, the computing device 200 may receive input data in severalseparate data packets or in a single data packet. For instance, datarepresenting rotation of the knob 312 and/or touch data may each bereceived via separate data packets or may all be received via the samedata packet, among others. Once the input data is received, some or allof the input data may be stored in a data storage (e.g., data storage204) and/or processed (e.g., using processors 202) to provide thefunctionality further discussed below.

At block 604, method 600 involves, in response to detecting the throttlemode input, setting an initial position of the knob in relation to thebase. The hand-holdable controller may perform a function to indicatethe initial position has been set. For example, a sonic output (i.e.,oscillating torque of the motor to produce a sound) may be used for ashort period of time after the initial position has been set.

In an example implementation, operating mode data refers to an operatingmode of the motor (e.g., motor 318) of the hand-holdable controller 300.Broadly, operating modes vary the speed and/or resistance of the controlknob and can be used individually or in combination with other operatingmodes. For example, in various operating modes, the motor can (i) varythe turning resistance of the knob (e.g., how hard it is to twist), (ii)rotate the control knob, (iii) prevent rotation of the knob, (iv) varythe speed at which the knob can turn, (v) vary the force that isrequired from the user to rotate the knob, (vi) vary the resistance ofthe knob to simulate detents (or ticks), (vii) to provide a return tocenter function (e.g., the motor is operable to return the knob to acertain position anytime a user releases the knob), (viii) to allow forcontinuous spin (e.g., the motor is set to operate in a free spin mode),(ix) to limit rotation (e.g., the motor is set to limit rotation towithin plus or minus 45 degrees from a center point), and/or (x)increase or decrease resistance the further the knob is rotated in onedirection (e.g., progressively increasing resistance as the knob isrotated may create the feeling of tension increasing as a component iswound up or a limit is reached).

Other operating modes (and combinations of operating modes) arepossible. For example, the hand-holdable controller may be configured toprovide a variable resistance through customizable arc sizes of varioussizes. For example, a full (or partial) rotation of the knob could bedivided into a variable number of arcs, and each arc could be of varioussizes. Each of the variable number of arcs could be defined to have aspecific feel (e.g., one or more operating modes, such as resistancelevels, speed, detents or no detents, etc.).

In an example implementation, the computing device 200 may send (e.g.,via communication link 402A) preconfigured or configurable throttle modedata for the knob 312 to the hand-holdable controller 300. The throttlemode data may be a pre-configured list, may be input by a user, may beloaded onto the computing device 200 (e.g., from an external memorystorage device), and/or may otherwise be input into the computing device200.

At block 606, method 600 involves detecting a movement of the knob 312in relation to the base 310. In some example implementations, thesensor(s) 314 may include a position sensor, such as a Hall effectsensor, that can determine the rotational position of the knob 312relative to the base 310.

At block 608, method 600 involves operating the at least one motor toapply torque-generating force to the knob 312 to return the knob 312 tothe initial position. Torque may be applied immediately after detectinga movement of the knob 312, after a period of time, when the throttlegrip is released, when a touch sensor is disengaged, or upon theoccurrence of some other triggering event.

Further, some form of feedback may be provided upon configuring thecomputing device 200 to operate the particular component. In oneexample, the computing device 200 may provide visual feedback (e.g., viadisplay 212) indicating the particular component being controlled. Inanother example, the computing device 200 may send a command to thehand-holdable controller 300 to cause vibrational feedback (e.g.,provided by the motor 318) such that a user holding the hand-holdablecontroller 300 can feel the vibration. In yet another example, visualfeedback may be provided by the particular component of the roboticsystem 100. For instance, an LED that is coupled to the particularcomponent may light up when the computing device 200 is set to operatethe particular component. Other examples may also be possible.

In an example implementation, the input data received by the computingdevice 200 may be interpreted, by the computing device 200, tocorrespond to a particular operation of the component that is based on(i) rotation of the knob and (ii) touch data received from the touchsensors during the rotation of the knob.

The throttle mode functionality of the hand-holdable controller 300 maydynamically change based on dynamically changing touch data. Forexample, as the hand-holdable controller 300 dynamically detects achange in touch data (e.g., a grip moving up or down on the controlleror from a full throttle grip with all four fingers to a partial throttlegrip with less than four fingers), the output commands may alsodynamically change (e.g., from a coarse adjustment with a full throttlegrip to a medium adjustment with a partial throttle grip or even to afine adjustment with a fingertip grip). Other examples may also bepossible.

In the case of a power-off event of a system such as the hand-holdablecontroller 300, the computing device 200 or the hand-holdable controller300 may store information related to a most recent configuration. Forinstance, if the hand-holdable controller 300 is operating in a throttlemode with particular functionality at the time of a power-off event, thehand-holdable controller 300 (or the computing device 200) may storeinformation related to the fact that this particular throttle mode wasthe most recent mode prior to the power-off event and/or may storeinformation related to characteristics of the particular operating mode,such as position of the knob 312. Subsequently, the hand-holdablecontroller 300 (or other component) can detect a power-on event. Uponsuch detection, the hand-holdable controller 300 may reconfigure tooperate using the most recent operating mode. For instance, thehand-holdable controller 300 may reconfigure to operate in the mostrecent operating mode prior to the power-off event. Other instances mayalso be possible.

IV. ADDITIONAL FEATURES

While control of the robotic system 100 was discussed above generally inthe context of controlling joints of the robotic system 100, suchdiscussion should not be seen as limiting as the example implementationsdiscussed herein may be used for control of a variety of differentrobotic system 100 components, as well as components/aspects of otherdevices and machines. In one example, rotation of the knob 312 may allowfor control of an internal combustion engine, an electric motor, aservomotor, or another component which may be usefully controlled by arotary twist-grip throttle mechanism. In another example, rotation ofthe knob 312 may allow for control of volume output of a speaker (e.g.,a speaker incorporated within the robotic system 100). In anotherexample, rotation of the knob 312 may allow for control of light outputfrom a light source (e.g., a light source incorporated within therobotic system 100). In yet another example, rotation of the knob 312may allow for control of movement/functionality of an end effector ofthe robotic system 100. Other examples may also be possible.

V. CONCLUSION

The present disclosure is not to be limited in terms of the particularimplementations described in this application, which are intended asillustrations of various aspects. Many modifications and variations canbe made without departing from its spirit and scope, as will be apparentto those skilled in the art. Functionally equivalent methods andapparatuses within the scope of the disclosure, in addition to thoseenumerated herein, will be apparent to those skilled in the art from theforegoing descriptions. Such modifications and variations are intendedto fall within the scope of the appended claims.

The above detailed description describes various features and functionsof the disclosed systems, devices, and methods with reference to theaccompanying figures. In the figures, similar symbols typically identifysimilar components, unless context dictates otherwise. The exampleimplementations described herein and in the figures are not meant to belimiting. Other implementations can be utilized, and other changes canbe made, without departing from the spirit or scope of the subjectmatter presented herein. It will be readily understood that the aspectsof the present disclosure, as generally described herein, andillustrated in the figures, can be arranged, substituted, combined,separated, and designed in a wide variety of different configurations,all of which are explicitly contemplated herein.

The particular arrangements shown in the figures should not be viewed aslimiting. It should be understood that other implementations can includemore or less of each element shown in a given figure. Further, some ofthe illustrated elements can be combined or omitted. Yet further, anexample implementation can include elements that are not illustrated inthe figures.

While various aspects and implementations have been disclosed herein,other aspects and implementations will be apparent to those skilled inthe art. The various aspects and implementations disclosed herein arefor purposes of illustration and are not intended to be limiting, withthe true scope being indicated by the following claims.

I claim:
 1. A system configured to operate a robotic device, the systemcomprising: a rotatable knob coupled to a base; at least one motor thatis operable to apply a torque-generating force to the rotatable knob; acurved touchpad comprising one or more touch sensors arranged to sensetouch input on a curved surface of the rotatable knob; and a controlsystem configured to: detect, based at least in part on touch data fromthe curved touchpad, a throttle mode input; and in response to thethrottle mode input, operate the at least one motor to affect therotation of the rotatable knob to simulate a throttle.
 2. The system ofclaim 1, wherein the throttle mode input further comprises inertial datafrom one or more sensors.
 3. The system of claim 1, wherein operation inthe throttle mode allows for rotation of the rotatable knob between: (i)an initial position and (ii) a stop position.
 4. The system of claim 1,wherein operation in the throttle mode further comprises operating theat least one motor to apply torque-generating force to the rotatableknob in accordance with a return-to-center function for the rotatableknob.
 5. The system of claim 3, wherein the control system is furtherconfigured to apply a sonic output, by oscillating torque through themotor, at the initial position and the stop position.
 6. The system ofclaim 3, wherein rotation from the initial position to the stop positionis in a first direction, and wherein operation in the throttle modefurther allows for rotation of the rotatable knob in a second directionfrom the initial position to a second stop position, wherein the seconddirection is opposite of the first direction.
 7. The system of claim 1,wherein the control system is further configured to (i) set an initialposition upon detecting the throttle mode input, (ii) detect a movementfrom the initial position, and (iii) in response to detecting themovement from the initial position, apply torque-generating force to therotatable knob to return the rotatable knob to the initial position. 8.The system of claim 1, wherein the control system is further configuredto apply a force-derivative torque to the knob in response to an appliedknob force, wherein the force-derivative torque is related to thederivative of the applied knob force.
 9. The system of claim 1, whereinthe control system is further configured to apply a velocity-dependentdamping torque to the knob in response to an applied knob force, whereinthe velocity-dependent damping torque is related to the velocity of theapplied knob force.
 10. The system of claim 9, wherein thevelocity-dependent damping torque further comprises a viscous dampingfactor.
 11. The system of claim 1, wherein the control system is furtherconfigured to apply a sonic output by oscillating torque through themotor.
 12. A method comprising: receiving touch data from a curvedtouchpad comprising one or more touch sensors, wherein the curvedtouchpad is arranged on a curved surface of a rotatable knob, wherein acontrol device comprises the rotatable knob coupled to a base, at leastone motor that is operable to apply torque-generating force to therotatable knob, and the curved touchpad; detecting, based at least inpart on the touch data, a throttle mode input; and in response to thethrottle mode input, operating the control device in a throttle mode,wherein operating the control device in a throttle mode compriseoperating at least one motor to affect the rotation of the rotatableknob to simulate a throttle.
 13. The method of claim 12, whereinoperating the control device in the throttle mode further comprisesapplying a return-to-center function to control rotation of therotatable knob.
 14. The method of claim 12, wherein detecting thethrottle mode input further comprises detecting inertial data from oneor more inertial sensors.
 15. The method of claim 14, wherein the touchdata indicates a throttle grip on the rotatable knob.
 16. The method ofclaim 12, wherein operation in the throttle mode allows for rotation ofthe rotatable knob between: (i) an initial position and (ii) a stopposition.
 17. A non-transitory computer readable medium comprisingprogram instructions executable by a processor to cause a computingdevice to perform operation comprising: receiving touch data from acurved touchpad comprising one or more touch sensors, wherein the curvedtouchpad is arranged on a curved surface of a rotatable knob, wherein acontrol device comprises the rotatable knob coupled to a base, at leastone motor that is operable to apply torque-generating force to therotatable knob, and the curved touchpad; detecting, based at least inpart on the touch data, a throttle mode input; and in response to thethrottle mode input, operating the control device in a throttle mode,wherein operating the control device in a throttle mode compriseoperating at least one motor to affect the rotation of the rotatableknob to simulate a throttle.
 18. The non-transitory computer readablemedium of claim 17, wherein causing the control device to operate in thethrottle mode further comprise operating the at least one motor toaccording to a return-to-center function.
 19. The non-transitorycomputer readable medium of claim 17, wherein detecting the throttlemode input further comprises detecting inertial data from one or moreinertial sensors.
 20. The non-transitory computer readable medium ofclaim 17, wherein the touch data indicates a throttle grip on therotatable knob.